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•  Background 

•  Description 

•  Application/Example 

•  Risk  Assessment  Tool 

•  Summary 


Turbine  Engine  “Building  Block” 
Technology  Demonstration  Process 


APPLIED  RESEARCH 
(6.2) 


ADVANCED  TECHNOLOGY  DEVELOPMENT  TECHNOLOGY  TRANSITION 

(6.3) 
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COMBUSTOR 


TURBINES 


MECHANICAL 

SYSTEMS 
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NOZZLE 


Seamless  Development  Process 


APSI JTDE  and  JETEC 
“ENGINE”  DEMONSTRATORS 


CONTROLS 


ATEGG  and  JTAGG  “CORE” 
TECHNOLOGY  DEMONSTRATORS 


Seamless  Contractor  Planning 


Technology  Readiness  Levels 


System  Test,  Flight 
and  Operations 


Sy  ste  m/S  u  bsy  ste  m 
Development 
(SDD) 

Technology  Demonstration 
(ATEGG/JTDE) 


Technology 

Development  (Rig  Testing) 

Research  to  Prove 
Feasibility 

Basic  Technology 
Research 


9  -  Actual  system  “Flight  Proven”  through 
successful  mission  operations 

8  -  Actual  system  completed  and  “Flight 
Qualified”  through  test  and  demonstration 

7  -  System  prototype  demonstration  in  an 
operational  environment 

6  -  System/Subsystem  model  or  prototype 
demonstration  in  a  relevant  environment 

5  -  Component  and  /  or  breadboard  validation 
in  relevant  environment 

4  -  Component  and  /  or  breadboard  validation 
in  laboratory  environment 

3  -  Analytical  and  experimental  critical  function 
and  /  or  characteristic  proof  -  of  -  concept 

2  -  Technology  concept  and  /  or  application 
formulated 


1  -  Basic  principles  observed  and  reported 


Test  Confidence  Rating  Purpose 


Test  Readiness  assessment  of  AFRL  6.3  Funded 
Advanced  Development  engine  programs 

(Engine  components,  instrumentation,  assembly  and  test 
facilities) 

Rating  of  program  at  key  program  milestones 
(Proposal  Eval,  PDR,  CDR,  TRR) 


Test  Confidence  Rating  Purpose 


•  Test  Readiness  assessment  of  AFRL  6.3  Funded 
Advanced  Development  engine  programs 

(Engine  components,  instrumentation,  assembly  and  test 
facilities) 

•  Rating  of  program  at  key  program  milestones 

(Proposal  Eval,  PDR,  CDR,  TRR) 

Note:  Program  ATD  programs 

1)  Have  signed  transition  plans 

2)  Use  James  Gregory  IPPD  process 


Test  Confidence  Rating  System 


APPROACH 

•  Use  1997  Component  Rating  Model  as  starting  point 

•  Review  R&D  Engine  past  problem  data  base 

•  Use  Guidance  from 

-  577th  AESG  Best  Practice  documents 

-  ENSIP  document  HCF  test  Protocol 

-  Existing  (FI 35,  FI 36,  FI  19,  etc)  System  Engineering  Program 

-  AFR  99-103  “Test  &  Evaluation” 

•  Benched  marked  model  against  previous  R&D  engines 


V’ROnJLSION  CENTER  OF 
EXCHiJENCE 


FEATURES  OF  RATING  SYSTEM 

“Exit  criteria”  at  Program  Kickoff,  PDR,  CDR,  hardware  delivery,  Test  Planning 
Hardware  responsibility  back  to  component  owner 
Component  level  risk  assessment  /  mitigation 
Review  of  manufacturing 
Review  of  inspection  records 
Review  of  instrumentation  &  assembly 
Review  of  test  facility  past  problems 


Agenda 
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•  Background 
>  Description 

•  Application/Example 

•  Risk  Assessment  Tool 

•  Summary 


Engine  Test  Confidence  Rating 

(TCR)  DESCRIPTION 

- - 


Expanded  prior  MERO  advanced  component  rating  process 

•  Instrumentation,  Assembly  &  Test  Materials 

•  Additional  Component  Design  Information  Environment 

■0  j  • 

•  Extensive  use  of  checklists  guide  rating  process  Quality 


Engine  Test  Confidence  Rating 

Component  Confidence  Rating 

Component  Confidence  Rating 
x  Engine  Assembly  / 

Instrumentation  Quality 

Test  Plan 

Instrumentation 

Test  Facility  /  Installation 
Special  Test  Equipment 

Component 

Material 

Manufacturing  Process 
Assembly  / 

Instrumentation  Quality 
Part  Quality 

TCR  Evaluation 


Demonstrator:  Silicon  Nitride  Blade  Example 

Date  of  Rating:  Now  Feb  07 
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Engine  Assembly  /  Instrumentation  Quality 
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Test  Plan 
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Output 

Instrumentation 
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Test  Facility  /  Installation 
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Special  Test  Equipment 
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Notes:  Must  Justify  rating  &  Identify  future  risk  reduction  efforts 

Agenda 

— 


•  Background 

•  Description 

>  Application/Example 

-  @  start  of  engine  testing 

•  Risk  Assessment  Tool 

•  Summary 


Engine  Example 


FJ44  for  Illustration  purposes  only-  FJ44  not  the  real  example 


Engine  Example 


FJ44  for  Illustration  purposes  only-  FJ44  not  the  real  example 


Turbine  Component  (C) 


0 

Idea! 

1 

Conceptual  Design 

2 

Preliminary  Design 

3 

Detailed  Design 

4 

Subcomponent  Rig  Tests 

5 

Subscale  Component  Rig  Test 

6 

Full  Scale  Component  Rig  Test 

7 

Demonstrator  Engine  Performance  T est 

8 

Demonstrator  Engine  Durability  Test 

9 

Demonstrator  Engine  Altitude  Test 

Rating  Criteria 


Conceptual  Design  Review  (CDR)  /  Kickoff  Meeting 

1.  Engine/component-level  goals/objectives  defined 
(performance,  efficiencies,  cooling  flows,  pressure  ratios,  etc.) 

2.  Initial  risk  assessment. 

3.  New  processes  identified. 

4.  Tech  Demo  Systems  Engineering  (TDSE)  deviations  identified, 
evaluated  and  addressed. 

5.  Test  facility,  facility  requirements,  preliminary  special  test  equipment 
and  safety  requirements  identified. 

6.  Initial  assessment  of  engine/component  environment 
(pressure,  temperatures  and  stresses). 

7.  Structural  Audit  format  defined 


Component  (C)  Rating  Criteria 

Preliminary  Design  Review  (PDR) 

1 .  Moderately  detailed  description  of  component  and  its  materials. 

2.  Acceptable  hardware  reworks,  changes,  and  refurbishment  since  previous  use 

3.  Sufficient  aerodynamic  and  mechanical  design  activity  to  allow 
all  long  lead  hardware  to  be  ordered. 

4.  Engine/component-level  goals/objectives  defined 
(performance,  efficiencies,  cooling  flows,  pressure  ratios,  etc.) 

5.  Risk  assessment  updated. 

6.  TDSE  deviations  identified,  evaluated  and  addressed. 

7.  Preliminary  manufacturing  plan  complete  (long  lead  hardware  identified). 

8.  Critical  or  new  manufacturing  processes/challenges  identified. 

9.  Test  facility,  facility  requirements,  preliminary  special  test  equipment 
and  safety  requirements  identified. 

10.  Preliminary  instrumentation  and  preliminary  assembly  plans  complete. 

1 1 .  Appropriate  lessons  learned  identified  and  incorporated. 

12.  Initial  Structural  Audit. 

13.  Updated  assessment  of  engine/component  environment 
(pressure,  temperatures  and  stresses). 


Rating  Criteria 


Detailed  Design  Review  (DDR) 

1 .  Pretest  performance  predictions  cover  all  key  test  points  . 

2.  Component  predicted  performance  and  operability  is  acceptable. 

3.  Final  assessment  of  engine/component  environment  (pressure,  temperatures  and  stresses) 

4.  Secondary  flow  analyzed  was  conducted  at  all  key  test  points. 

5.  Acceptable  data  acquisition  and  safety  monitoring,  and  all  critical  limits  are  defined. 

6.  Acceptable  Instrumentation  features/routing  to  include  changes  from  previous  builds. 

7.  Critical  pieces  of  instrumentation  have  back-ups. 

8.  Yellow  and  red  limits  are  defined  for  all  safety  critical  parameters 
(speeds,  vibration,  temperatures,  pressures,  calculated  parameters,  etc). 

9.  All  clearances  (compressor  &  turbine  tip,  etc)  are  consistent  with  test  points 

10.  Blade  and  vane  vibratory  responses  (Campbells  &  Goodmans)  are  acceptable 

1 1 .  High  Cycle  Fatigue  test  protocol  has  been  applied). 

12.  Critical  or  new  manufacturing  processes/challenges  identified. 

13.  Test  facility,  facility  requirements,  preliminary  special  test  equipment 

14.  Instrumentation  and  assembly  plans  updated. 

15.  Appropriate  lessons  learned  identified  and  incorporated. 

1 6.  Known  risks  have  been  addressed. 

17.  Appropriate  TDSE  deviations  identified  and  addressed. 


Silicon  Nitride  Material  (M) 


0 

Unattainium! 

1 

Initial  Coupon  data 

2 

Coupon  data  with  some  extrapolation 

3 

Coupon  data  at  relevant  conditions 

4 

Subcomponent  data  with  extrapolation 

5 

Subcomponent  data  with  interpolation 

6 

Subcomponent  data  at  relevant  engine  test  conditions  (1-2 
data  points) 

7 

Subcomponent  data  at  relevant  engine  test  conditions  (3+ 
data  points) 

8 

-la  data 

9 

-3a  production  values 

Manufacturing  Process  (MP) 


0 

Idea! 

1 

Unproven  process 

2 

Nonvalidated  inspection  of  unproven  process 

3 

Process  feasibility  demonstrated 

4 

Nonvalidated  inspection  of  demonstrated  process 

5 

Proof  spin  of  demonstrated  process  at  relative  loads 

6 

Validated  inspection  of  demonstrated  process  or  cyclic 
life  test  of  demonstrated  process 

7 

Validated  inspection  and  prior  engine  test  of 
demonstrated  process 

8 

Validated  inspection  of  production  process 

9 

Production  inspection  of  production  process 

Turbine  Assembly  /  Instrumentation 

Quality  (AIQ) 


0 

No  inspection  and  sign  off  (I&S  0) 

1 

Third  tier  subcontractor  component  owner  I&SO 

2 

Second  tier  subcontractor  component  owner  I&SO 

3 

Subcontractor  component  owner  I&S  0 

4 

Original  Engine  Manufacture  (OEM)  component  owner  component 
I&SO 

5 

OEM  component  owner  component  and  subassembly  I&S  0 

6 

OEM  component  owner  component,  subassembly  and  part  I&S  0  or 
previously  successful  engine  test  if  not  disassembled  or  TDSE  plan 
met  and  all  CDR,  PDR,  DDR  and  TRR  requirements  are  met 

7 

Successive  build  experience  (second  build) 

8 

Successive  build  experience  (2+  builds) 

9 

Innovative  quality  control  procedures  to  reduce  risk  (6a  process) 

Turbine  Part  Quality  (PQ) 


0 

No  inspection  and  sign  off  (I&S  0) 

1 

Third  tier  subcontractor  I&SO 

2 

Second  tier  subcontractor  I&SO 

3 

Part  and  process  (casting,  hole  drilling,  weld,  braze  etc.)  level 
subcontractor  component  owner  I&S  0 

4 

OEM  review  of  manufacturing  inspection  records 

5 

OEM  visual  review  of  parts  and  manufacturing  inspection  records 

6 

OEM  component  owner  visual  review  of  parts  and  manufacturing 
inspection  records  or  previously  successful  engine  test  if  not 
disassembled  or  TDSE  plan  met  and  all  CDR,  PDR,  DDR  and 
HDTOEM  requirements  are  met 

7 

8 

9 

Innovative  quality  control  procedures  to  reduce  risk  (6a  process) 

0 

No  inspection  and  sign  off  (I&S  0) 

1 

Second  tier  subcontractor  assembler  /  technician  I&SO 

2 

Subcontractor  assembler  /  technician  I&SO 

*3 

OEM  assembler  /  technician  I&S  0 

4 

5 

OEM  component  owner  I&SO  of  high  risk  components 

6 

OEM  component  owner  I&S  0  or  TDSE  plan  met  and  all  CDR,  PDR, 
DDR  and  TRR  requirements  are  met 

7 

Successive  build  experience  (second  build) 

8 

Successive  build  experience  (2+  builds) 

9 

Innovative  quality  control  procedures  to  reduce  risk  (6a  process) 

Test  Plan  (TP) 


0 

No  requirements  addressed 

1 

Some  PDR  requirements  met 

2 

PDR  requirements  met 

3 

PDR  requirements  exceeded 

4 

PDR  and  DDR  requirements  met 

5 

PDR,  DDR  and  TRR  requirements  are  met 

6 

PDR,  DDR,  TRR  and  AEI  requirements  1-17  met 

7 

PDR,  DDR,  TRR  and  AEI  requirements  1-18  met 

8 

PDR,  DDR,  TRR  and  AEI  requirements  exceeded 

9 

Innovative  test  planning  techniques  to  reduce  risk 

Engine  Instrumentation  (I) 


0 

No  requirements  addressed 

1 

Some  PDR  requirements  met 

2 

PDR  requirements  met 

3 

PDR  requirements  exceeded 

4 

PDR  and  DDR  requirements  met 

5 

PDR  and  DDR  requirements  exceeded 

6 

PDR,  DDR,  and  SMPTET  requirements  met 

7 

PDR,  DDR,  and  SMPTET  requirements  exceeded 

8 

PDR,  DDR,  and  SMPTET  requirements  exceeded  with 
some  first  generation  advanced  instrumentation 

9 

PDR,  DDR,  and  El  requirements  exceeded  with  some 
second  generation  advanced  instrumentation 

est  Facility  /  Installation  (TFI) 


0 

No  requirements  addressed 

1 

Some  PDR  requirements  met 

2 

PDR  requirements  met 

3 

PDR  requirements  exceeded 

4 

PDR  and  DDR  requirements  met 

5 

PDR,  DDR,  and  SMPTET  requirements  met 

6 

PDR,  DDR,  SMPTET,  and  AEI  requirements  met 

7 

PDR,  DDR,  SMPTET,  and  AEI  requirements  exceeded 

8 

Successive  test  facility  experience  (second  build) 

9 

Successive  test  facility  experience  (2+  builds) 

Special  Test  Equipment  (STE) 


0 

No  requirements  addressed 

1 

Required  STE  identified  (slip  ring,  oil  cart,  etc) 

2 

STE  specifications  identified  (channels,  flow,  etc) 

3 

STE  PDR  complete 

4 

DDR  requirements  met 

5 

DDR  requirements  exceeded 

6 

DDR  and  SMPTET  requirements  met 

7 

DDR  and  SMPTET  requirements  exceeded 

8 

Successive  build  experience  (second  build) 

9 

Successive  build  experience  (2+  builds) 

W  TCR  Calculation 


Is  risk  at  test  acceptable? 


Component  Confidence  Rating 

Component 

=  5 

Material 

=  6 

Manufacturing  Process 

=  5 

Assembly  /  Instrumentation  Quality 

=  3 

Part  Quality 

=  3 

cc  =  Aj  C*M*MP*AIQ*PQ 

AJ  5*6*5*3*3  =  4.2 

Test  Confidence  Rating 

Component  Confidence  Rating 

=  4.2 

Engine  Assembly  /  Instrumentation  Quality  =  3 

Test  Plan 

=  6 

Instrumentation 

=  6 

Test  Facility  /  Installation 

=  9 

Special  Test  Equipment 

=  9 

TCR  =  /\J  CC*EAIQ*TP*I*TFI*STE 

-  /V  42*3*6-6*9*9  .  5.8 

TCR  Evaluation 


TCR=5.8  Not  acceptable  risk,  need  TCR>6 

Action: 

C=6  Full  scale  turbine  aero  rig  test 
MP=6  Cyclic  life  spinpit  testing 
AIQ=6  Component  owner  is  part  of  Component 
PQ=6  Assembly,  Inspection  ,  and  Engine 

EAIQ=6  Assembly 


cc 

TCR 


C*M*MP*AIQ*PQ 


CC*EAIQ*TP*I*TFI*STE 


TCR  Evaluation 


Demonstrator:  Silicon  Nitride  Blade  Example 

Date  of  Rating:  Now  Feb  07 
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Engine  Assembly  /  Instrumentation  Quality 
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Output 

Instrumentation 
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Test  Facility  /  Installation 
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Notes:  Must  Justify  rating  &  Identify  future  risk  reduction  efforts 

Agenda 

— 


•  Background 

•  Description 

•  Application/Example 
>Risk  Assessment  Tool 

•  Summary 


Consequence  Likelihood 


Engine  TCR  Can  Quantify  Risks* 


Level 


Level 


Likelihood 


Not  Likely 


Low  Likelihood 


Likely 


Highly  Likely 


Near  Certainty 


Probability  of  Occurrence 


-10% 


-30% 


-50% 


-70% 


-90%. 


Technical 


Minimal  or  no  impact 


Minor  technical  shortfall, 
no  impact  to  high  level 
technical  requirements 


Moderate  technical  shortfall 
but  work  around  available 
which  will  eliminate  impact 
to  high  level  technical 
requirements 


Unacceptable,  work 
arounds  available  which 
will  eliminate  impact  to 
high  level  technical 
requirement 


Schedule 


Minimal  or  no  impact 


Additional  activities  required,  able  to 
meet  key  dates. Slip  < month(s) 


Minor  schedule  slip,  no  impact  to  key 
milestones. 

Slip  < _ month(s)  of  critical  path. 

Sub-system  slip  > _ month(s). 


Program  critical  path  affected,  all 
schedule  float  associated  with  key 
milestone  exhausted  Slip  <  months 


2  3  4  5 

Seauence 

Cost 

Minimal  or  no  impact 


Budget  increase  or  unite 

production  cost  increases  < _ (1% 

of  Budget) 


Budget  increase  or  unit 

production  cost  increase  < _ (5% 

of  Budget) 


Budget  increase  or  unit 

production  cost  increase  < _ (10% 

of  Budget) 


*Risk  Management  Guide  For  DOD  Acquisition,  Jun  03,  DOD  DAU 


Agenda 

— 


•  Background 

•  Description 

•  Application/Example 

•  Risk  Assessment  Tool 
>  Summary 


Summary 


TCR  developed  for  Turbine  Engines 

•  Applied  at  key  program  milestones 

-  Program  Award,  PDR,  CDR  &  test 

•  Evaluates  test  readiness  of  engine  components, 
instrumentation,  assembly  and  test  facilities 

•  Establishes  quantitative  risk  assessment 

•  Engine  TCR  is  flexible  and  could  be  tailored  to 
be  applicable  across  many  technical  areas 


